This is a quantized version of TheDrummer's Magidonia-24B-v4.3 large language model. By using the imatrix quantization technology of llama.cpp, GGUF format files with various precisions from BF16 to IQ2_XS are generated, aiming to run the model more efficiently under different hardware conditions (especially in resource-constrained environments) while maintaining the model performance as much as possible.

Natural Language ProcessingGguf

This is the GGUF quantized version of the large language model Mistral-Large-3-675B-Instruct-2512 developed by Mistral AI. The original model has 675 billion parameters and is designed for instruction-following tasks. This project uses the llama.cpp tool and combines it with the imatrix calibration dataset to generate more than 20 different precision quantized model files from Q8_0 to IQ1_S, aiming to balance model performance, inference speed, and storage/memory usage, enabling it to run on a wider range of hardware.

Natural Language ProcessingGgufMultiple Languages

This is a quantized version of the Microsoft Fara-7B model, optimized using the imatrix technology of llama.cpp. The model supports multiple quantization levels, from high-precision bf16 to extremely low-precision IQ2_M, meeting the needs of different hardware configurations and performance requirements.

Natural Language ProcessingGguf

This is a quantized version of TheDrummer's Snowpiercer-15B-v4 model, quantized using llama.cpp and imatrix technology. This version offers multiple quantization levels from BF16 to IQ2_S, meeting the needs of users with different hardware conditions and performance requirements. The model is calibrated and optimized using a specific dataset and supports the online repackaging function to improve performance on ARM and AVX hardware.

Natural Language ProcessingGguf

This is the GGUF quantized version of NVIDIA's Qwen3-Nemotron-32B-RLBFF large language model. It uses the llama.cpp tool for multi-precision quantization, offering more than 20 quantization options from BF16 to IQ2_XXS, suitable for different hardware configurations and performance requirements.

Natural Language ProcessingGguf

This is a quantized version of the Ling-mini-2.0 model from inclusionAI, quantized using the imatrix technology of llama.cpp. It offers various quantization options from 32.54GB BF16 to 4.66GB IQ2_XS and can run efficiently in multiple hardware environments.

Natural Language ProcessingGguf

This project is a quantized version of the TECHNOPRAVIN01/Qwen2.5-14B-Valor model, providing model files in various quantization formats, including multiple quantization types such as IQ1_S, IQ2_XS, IQ3_M, Q4_K_S, Q5_K_M, etc., which are convenient for use in different scenarios.

Natural Language ProcessingTransformersEnglish

This is a quantized version of the Beck-4B model by gustavecortal. It uses the llama.cpp tool and the imatrix dataset for quantization optimization, providing a variety of quantization types from BF16 to IQ2_M, and can run efficiently in multiple hardware environments.

Natural Language ProcessingGguf

A quantized version of GLM-4.6 optimized based on the ik_llama.cpp branch. It uses advanced IQ quantization technology to significantly reduce memory usage while maintaining high-quality output. This series includes various quantization levels, from IQ5_K to IQ1_KT, to meet the inference needs in different hardware environments.

Natural Language ProcessingGguf

GLM 4.6 is a quantized model optimized for a configuration of 128GB memory + single GPU. It uses the IQ_K quantization method, providing better quality and performance than the mainstream llama.cpp at the same size. This model needs to be used with ik_llama.cpp. On a configuration of 128GB dual-channel DDR5 memory, a single CCD Ryzen 7000 processor + a single 3090 graphics card, the text generation speed can reach approximately 6.8 tokens per second.

Natural Language ProcessingGgufMultiple Languages

This project is the result of quantizing the zenlm/zen-eco-4b-instruct model, providing model files in multiple quantization versions, including various quantization levels such as IQ1_S, IQ2_XXS, IQ3_S, Q4_K_S, Q5_K_S, etc., which are convenient for users with different hardware configurations and performance requirements.

Natural Language ProcessingTransformersEnglish

GLM-4.5-Base is a text generation model that has undergone quantization processing. It is optimized based on the zai-org/GLM-4.5-Base model and is specifically designed for devices with 128GB of memory and a small GPU. It uses the new IQ2_KL quantization method of ik_llama.cpp, significantly reducing resource requirements while maintaining performance, and is suitable for text generation tasks.

Natural Language ProcessingGgufMultiple Languages

A quantized version of AGI-0 Art-0-8B, optimized using the imatrix technology of llama.cpp, which can run efficiently under different hardware conditions. It offers multiple quantization levels to choose from, ranging from high-quality Q8_0 to lightweight IQ2_M, meeting different memory and performance requirements.

Natural Language ProcessingGguf

This is the Llamacpp imatrix quantization version of the NVIDIA Nemotron-Nano-12B-v2 model, offering a variety of quantization options, from BF16 to IQ2 quantization with extremely low bit rates, to help users run the model efficiently under different hardware conditions.

Natural Language ProcessingGguf

This is a quantized version of TheDrummer's Behemoth-X-123B-v2 model. It uses the llama.cpp tool and a specific dataset for imatrix quantization, offering various quantization types from Q8_0 to IQ1_S, which can run efficiently in different hardware environments.

Natural Language ProcessingGguf

This is a quantized version of the Seed-OSS-36B-Instruct model from ByteDance-Seed. It undergoes multi-precision quantization processing using the llama.cpp tool, offering more than 20 quantization options from BF16 to IQ2_XXS, aiming to enhance the model's operating efficiency and performance on different hardware.

Natural Language ProcessingGguf

A quantized version in GGUF format based on the DeepSeek-V3.1 model, using the ik_llama.cpp branch for optimal quantization. This model achieves the best perplexity performance under a specific memory footprint and offers various quantization schemes from IQ5_K to IQ1_S to meet different memory and performance requirements.

Natural Language ProcessingGgufOther

This is a quantized version of the DeepSeek-V3.1 model, which uses the imatrix technology of llama.cpp for quantization processing. The aim is to improve the operating efficiency and performance of the model on different hardware. This version offers multiple quantization levels, ranging from high-quality Q8_0 to extremely low-quality IQ1_M, to meet the requirements of different hardware configurations and performance needs.

Natural Language ProcessingGguf

This is a quantized version of the zai-org GLM-4.5 model, implemented through the quantization technology of llama.cpp, which can run efficiently under different hardware conditions. It offers a variety of quantization levels from Q8_0 to IQ1_S and supports text generation tasks. The model file size ranges from 77GB to 380GB.

Natural Language ProcessingGguf

This is the quantized version of the aws-prototyping/codefu-7b-v0.1 model. It uses the llama.cpp tool for imatrix quantization processing and provides various quantization options from bf16 to IQ2_M, suitable for different hardware environments and performance requirements.

Natural Language ProcessingGguf